Skip to content

feat: Resolve available, non-secret environment variables during compile#2118

Merged
ferenc-csaky merged 4 commits into
mainfrom
feat/secret-env-var-resolve
Jun 10, 2026
Merged

feat: Resolve available, non-secret environment variables during compile#2118
ferenc-csaky merged 4 commits into
mainfrom
feat/secret-env-var-resolve

Conversation

@ferenc-csaky

@ferenc-csaky ferenc-csaky commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Key Changes

  • Add SecretEnvVarResolver that will resolve available regular env var definitions, and defines a new secret env var template ${{...}} that will be resolved to ${...} in `compile
  • compile env var resolution is applied for
    • The table properties of any Flink CREATE TABLE statements
    • Leaf nodes under the connectors key in any user-provided package.json

More detailed behavior explanation is added to the docs.

TODO

  • Requires a change in the runner first
  • Also resolve connector config options
  • Update docs and explain secret env vars

@ferenc-csaky ferenc-csaky force-pushed the feat/secret-env-var-resolve branch from efff74b to 32b7477 Compare June 8, 2026 12:52
@ferenc-csaky ferenc-csaky force-pushed the feat/secret-env-var-resolve branch from 32b7477 to 6e41c4c Compare June 8, 2026 18:56
@ferenc-csaky ferenc-csaky marked this pull request as ready for review June 8, 2026 18:57
@ferenc-csaky ferenc-csaky force-pushed the feat/secret-env-var-resolve branch from d4917b6 to 035719e Compare June 9, 2026 17:35
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 44.57831% with 46 lines in your changes missing coverage. Please review.
✅ Project coverage is 14.04%. Comparing base (581f403) to head (faea062).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
.../engine/stream/flink/plan/FlinkSqlNodeFactory.java 0.00% 30 Missing ⚠️
...main/java/com/datasqrl/util/ConfigLoaderUtils.java 64.51% 9 Missing and 2 partials ⚠️
.../com/datasqrl/planner/Sqrl2FlinkSQLTranslator.java 0.00% 2 Missing ⚠️
...datasqrl/planner/util/NonSecretEnvVarResolver.java 89.47% 1 Missing and 1 partial ⚠️
...com/datasqrl/planner/tables/FlinkTableBuilder.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2118      +/-   ##
============================================
+ Coverage     13.92%   14.04%   +0.11%     
- Complexity      854      864      +10     
============================================
  Files           604      606       +2     
  Lines         17252    17348      +96     
  Branches       2086     2103      +17     
============================================
+ Hits           2403     2436      +33     
- Misses        14617    14677      +60     
- Partials        232      235       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ferenc-csaky ferenc-csaky force-pushed the feat/secret-env-var-resolve branch from 035719e to 450a7b4 Compare June 9, 2026 19:09

@mbroecheler mbroecheler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The structure and setup make sense to me. Nicely done!
Do we have test coverage that an env var which is not available at compile time is left in place?

@ferenc-csaky ferenc-csaky enabled auto-merge (squash) June 10, 2026 11:10
@ferenc-csaky ferenc-csaky merged commit 5c9e9f9 into main Jun 10, 2026
14 checks passed
@ferenc-csaky ferenc-csaky deleted the feat/secret-env-var-resolve branch June 10, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants